An experiment in agent teaching by subject matter experts

This paper presents a successful knowledge acquisition experiment in which subject matter experts that did not have any prior knowledge engineering experience succeeded to teach the Disciple-COA agent how to critique courses of action, a challenge problem addressed by the DARPA’s High Performance Knowledge Bases program. We first present the COA challenge problem and the architecture of Disciple-COA, a learning agent shell from the Disciple family. Then we present the knowledge acquisition experiment, detailing both the expert-Disciple interactions, and the automatic knowledge base development processes that take place as a result of these interactions. The results of this experiment provide strong evidence that the Disciple approach is a viable solution to the knowledge acquisition bottleneck.